Expl(Ai)Ned: The Impact of Explainable Artificial Intelligence on Cognitive Processes

This paper explores the interplay of feature-based explainable AI (XAI) techniques, information processing, and human beliefs. Using a novel experimental protocol, we study impact providing users with explanations about how an system weighs inputted to produce individual predictions (LIME) on users' weighting beliefs task-relevance information. On one hand, find that cause alter their mental available according observed explanations. other lead asymmetric belief adjustments interpret as manifestation confirmation bias. Trust in prediction accuracy plays important moderating role for XAI-enabled adjustments. Our results show XAI does not only superficially influence decisions but really change internal cognitive processes, bearing potential manipulate reinforce stereotypes. Hence, current regulatory efforts aim at enhancing algorithmic transparency may benefit from going hand measures ensuring exclusion sensitive personal systems. Overall, our findings put assertions is silver bullet solving all systems' (black box) problems into perspective.